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INTRODUCTION 



THE NATION’S 
REPORT 
CARD 

1992 

Trial State Assessment 




The National Assessment of Educational Progress (NAEP) is a Congressionally mandated 
project of the National Center for Education Statistics (NCES) that has collected and 
reported information for nearly 25 years on what American students know and what they 
can do. It is the nation’s only ongoing, comparable, and representative assessment of 
student achievement. Its assessments are given to scientifically selected samples of youths 
attending both public and private schools and enrolled in grades four, eight, or twelve. The 
assessment questions are written around a framework prepared for each content area — 
reading, writing, mathematics, science, and others — that represents the consensus of groups 
of curriculum experts, educators, members of the general public, and user groups on what 
should be covered on such an assessment. Reporting includes means and distributions of 
scores, as well as more descriptive information about the meaning of the data. 



New Reading Assessment Framework and Questions 



The goal of the National Center for Education Statistics is to make data available for the 
public and to do so in accurate and understandable ways that are not misleading. The task 
is challenging because much of what matters in NAEP is changing: 



• the content in response to the developing standards of various curricular 
groups; 

• the assessment questions in response to new developments in assessments; 
and 

• the reporting in response to increasing interest in student achievement 
relative to standards of student performance. 



O 

ERIC 




THE 1992 NAEP TRIAL STATE ASSESSMENT 



1 



Utah 



The framework for the 1992 Trial State Assessment Program in reading considered 
students’ performance in situations that involved reading different kinds of materials for 
different purposes. The fourth-grade reading assessment measured two global purposes for 
reading — reading for literary experience and reading to gain information. (The eighth- and 
twelfth-grade national NAEP reading assessments also measured a third purpose for 
reading — reading to perform a task.) Reading for literary experience usually involves the 
reading of novels, short stories, plays, and essays. In these reading situations, the reader 
can determine how the author explores or uncovers experiences through the text and 
considers the interplay among events, emotions, and possibilities. Reading to gain 
information usually involves the reading of articles in magazines and newspapers, chapters 
in a textbook, entries in encyclopedias and catalogs, and entire books on particular topics. 
These reading situations call for different orientations to text from those in reading for 
literary experience because readers are specifically focused on acquiring information. 

The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 

• Initial Understanding - comprehending the overall or general meaning of 
the selection. 

• Developing an Interpretation - extending the ideas in the text by making 
inferences and connections. 

• Personal Response — making explicit connections between ideas in the text 
and a student’s own background knowledge and experiences. 

• Critical Stance — considering how the author crafted a text. 

These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. 

The 1992 NAEP reading assessment uses a variety of innovative assessment approaches 
that are considered significant advancements over previous assessments. In addition to 
multiple-choice questions, the assessment primarily includes constructed-response 
questions that ask students to demonstrate comprehension beyond a surface level. Also, 
longer and naturally-occurring reading materials are used to provide more realistic reading 
experiences than in previous assessments. 

Taken together, the changes in the 1992 reading framework and assessment activities 
preclude any comparisons between the results in this report and those for previous NAEP 
reading assessments. If the current NAEP framework is used in the future, as planned in 
the 1994 assessment, the 1992 reading data will supply the basis for a trend report. 
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A Transition in Reporting 

Over time there have been many changes in emphasis of NAEP reporting, both to take 
advantage of new technologies and to reflect changing trends in education. In 1984, a new 
technology called Item Response Theory (IRT) made it possible to create “scale scores” 
for NAEP similar to those the public was accustomed to seeing for the annual Scholastic 
Aptitude Test (SAT). Educational Testing Service, in its role as Government grantee 
carrying out NAEP operations, devised a new way to describe performance against this 
scale, called “anchor levels.” Starting in 1984, NAEP results were reported by “anchor 
levels.” Anchor levels describe distributions of performance at selected points along the 
NAEP scale (i.e., standard deviation units). Anchor levels show how groups of students 
perform relative to each other, but not whether this performance is adequate. 

In 1988, Congress established the National Assessment Governing Board (NAGB), 
assigning it broad policy making authority over NAEP, including the authority to take 
“appropriate actions ... to improve the form and use of the National Assessment” and to 
identify “appropriate achievement goals for each . . . grade and subject area to be tested in 
the National Assessment.” To carry out its responsibilities, NAGB developed 
“achievement levels,” which are collective judgments about how students should perform 
relative to a body of content reflected in the NAEP frameworks. The result is translated 
onto ranges along the NAEP scale. For the 1992 reading assessment, this process was 
conducted for NAGB under contract by American College Testing (ACT), which has 
extensive experience in standard -setting in many fields. 

With this background, the initial reports for the 1992 reading assessment mark NCES’s 
continued attempt to shift to standards-based reporting of National Assessment statistics. 
The first transition to reporting NAEP results by achievement levels was for the NAEP 
1992 Trial State Assessment in mathematics. 1 The impetus for this transition lies in the 
belief that NAEP data will take on more meaning for the public if they show what 
proportion of our youth are able to meet judgmental standards of performance. 

Reporting NAEP results on the basis of achievement levels represents a significant change 
in practice for NCES. On occasion, this agency makes use of emerging analytical 
approaches that permit new, and sometimes controversial, analyses to be done. When 
doing so, this agency, just as other statistical agencies do when introducing new measures 
to supplement or replace old measures, also has provided the data according to the earlier 
procedures in addition to the new ones. In the case of the 1992 mathematics assessment, 
for example, the “anchor levels” or “scale anchoring” method of reporting was presented 
in an appendix. 



1 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993) and the individual 
1992 Mathematics State Reports. 
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In this assessment, the “scale anchoring” methodology used by NAEP since 1985 has been 
used but in a new way. As implemented for this report, the scale anchoring process applies 
not to regular scale intervals (standard deviation units), but to the achievement levels 
established for fourth-grade students. 2 The details of this procedure are presented in 
Appendix D. The critical distinction here is that setting achievement levels attempts to 
describe what students should be able to do in various ranges of the NAEP scale while the 
anchoring procedure attempts to describe what they can do at those achievement levels 
using actual student performance data from the NAEP assessments. 

Chapter 1 of this report describes how the 1992 standards were prepared and provides 
examples of assessment questions that illustrate the reading content reflected in the 
descriptions of the NAEP achievement levels. Chapters 1 - 6 include information on 
overall means, distributions of reading proficiency, as well as background questionnaire 
data, all taken directly from the results of the assessment questions. 

Continuing Development Effort 

We believe that the numerous completed and ongoing studies 3 will lead to national debate 
that can assure the public is well informed about these issues — as informed they must be 
because the results will be a vital influence on what Americans come to think about the 
condition and progress of our schools. Indeed, measures of student learning may be as 
significant bases for public understanding about our nation’s education system as the 
Consumer Price Index and the monthly unemployment statistics are in informing the 
public about our nation’s economy. 

In addition, members of the public need the data in this report to see for themselves what 
standards-based reporting might do and to evaluate the often conflicting claims of adherents 
and detractors of these changes in approaches to reporting on the educational achievement 
of American students. Reporting NAEP results to the public would be more clear if the 
language of the achievement levels, or standards, could also directly describe what students 
know and can do. In order to accomplish that, the frameworks, assessment questions, and 
achievement levels may need to be developed in tandem. That is easier to say than to do, 
however, because it implies a substantially larger pool of assessment questions, carefully 
designed to support reporting about performance relative to a set of performance standards. 
Clearly this is a developmental effort that will take time and several iterations, during which 
data supporting appropriate inferences about the performance of American students will 
be gathered on a continuing basis. 



2 First, students were identified who performed at or around the three achievement levels on the scale (212, 243, 
and 275). Next, questions were identified that were answered correctly by 65 percent or more of the 
fourth-grade students at the cutpoint for that achievement level. Finally, reading educators were asked to 
analyze each anchor-level question and create summary descriptions of the skills and abilities evidenced by 
students who answered these sets of questions successfully. 

Educational Achievement Standards : Setting Achievement Levels for the Nation. The Second Report of the 
National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: 1992 Trial 
State Assessment. (Stanford, CA: National Academy of Education, 1993).; U.S. General Accounting Office 
(GAO). Educational Achievement Standards: NAGB's Approach Yields Misleading Interpretations. June 
1993. GAO/PEMD-93-12.; Assessing Student Achievement in the States. The First Report of the National 
Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment : 1990 Trial State 
Assessment. (Stanford, CA: National Academy of Education, 1992).; R.L. Linn, D.M. Koretz, E.L. Baker, 
and L. Burstein. The Validity and Credibility of the Achievement Levels for the 1990 National Assessment of 
Educational Progress in Mathematics. (Los Angeles, CA: Center for Research on Evaluation, Standards, and 
Student Testing, UCLA, June 1991) CRESST Report 330. 
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A Recent History of NAEP 

In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by-state assessments on a trial basis. 

As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public-school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 4 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public-school students assessed in 41 states, the District of 
Columbia, and two territories. 5 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 

School and Student Participation in the Reading Assessment 

In Utah, 108 public schools participated in the fourth-grade reading assessment. The 
weighted school participation rate was 99 percent, which means that the fourth-grade 
students in this sample of schools were directly representative of 99 percent of all the 
fourth-grade public-school students in Utah. 




4 For a summary of the 1990 program, see Ina V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement: NAEP’s 1990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 

5 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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In total, 2,829 fourth-grade Utah public-school students were assessed. The weighted 
student participation rate was 96 percent. This means that the sample of students who 
took part in the assessment was directly representative of 96 percent of the eligible 
fourth-grade public-school student population in participating schools in Utah (that is, all 
students from the population represented by the participating schools, minus those students 
excluded from the assessment). 

The overall weighted response rate (school rate times student rate) was 95 percent. This 
means that the sample of students who participated in the assessment was directly 
representative of 95 percent of the eligible fourth-grade public-school student population 
in Utah. 



Students’ Reading Performance 



As shown in the following figure, the overall average proficiency of fourth-grade 
public- school students from Utah on the NAEP reading scale was 222. This proficiency . 
was higher than that of students across the nation (2 16). 6 There also was a tremendous 
range in student performance. The lowest performing 10 percent of the fourth graders from 
Utah had proficiency levels below 180 while the top 10 percent of the fourth graders had 
proficiency levels above 260. 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency 




Trial State Assessment 




The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by HH). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 



6 



Differences reported as significant are statistically different at the 95 percent confidence level. This means that 
with 95 percent confidence there is a real difference in the average reading proficiency between the two 
populations of interest. “About the same” means that no statistically significant difference was found at the 
95 percent confidence level. . . 
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LEVELS OF ACHIEVEMENT 

When Congress established the National Assessment Governing Board (NAGB) in 1988 
to set policy for NAEP, it charged the board with “identifying appropriate achievement 
goals for each age and grade in each subject area to be tested under the National 
Assessment.” (Pub. L. 297-100 Section 3403 (a)(5)(B)(ii)). 

NAGB developed three achievement levels for each grade — Basic, Proficient, and 
Advanced. Performance at the Basic level denotes partial mastery of the knowledge and 
skills that are fundamental for proficient work at each grade level. The central level, called 
Proficient, represents solid academic performance at each grade level tested. Students 
reaching this level demonstrate competency over challenging subject matter and are well 
prepared for the next level of schooling. Achievement at the Advanced level signifies 
superior performance at the grade tested. Definitions of the three levels of reading 
achievement are given below. 



BASIC l. : 

^ 'tiviL % H 


Fourth-grade students performing at the Basic level should demonstrate an 
understanding of the overall meaning of what they read. When reading texts 
appropriate for fourth graders, they should be able to make relatively obvious 
connections between the text and their own experiences. 


PROFICIENT 
k. LEVEL • fcJ 
■f ’%4§) ? ; 


Fourth-grade students performing at the Proficient level should be able to 
demonstrate an overall understanding of the text, providing inferential as well 
as literal information. When reading text appropriate to fourth grade, they 
should be able to extend the ideas in the text by making inferences, drawing 
conclusions, and making connections to their own experiences. The connection 
between the text and what the student infers should be clear. 


^ADVANCE©- 

LEVEL 

|£" T(27fjf 


Fourth-grade students performing at the Advanced level should be able to 
generalize about topics in the reading selection and demonstrate an awareness 
of how authors compose and use literary devices. When reading text 
appropriate to fourth grade, they should be able to judge texts critically and, in 
general, give thorough answers that indicate careful thought. 



Because the process of setting the levels of reading achievement centered on the descriptions 
of what students should be able to do, it is important to explore whether students actually 
met the expectations for performance at the Basic, Proficient, and Advanced levels. To 
help in this process, NCES arranged for ETS to apply a modified anchoring procedure to 
the 1992 reading achievement levels. A committee of reading education experts was 
assembled to review the questions and assessment results. Using their knowledge of reading 
and student performance on the individual questions, the committee members were asked 
to summarize student performance at each achievement level (see Appendix D for more 
details on the anchoring procedure). 

Placing the descriptions of how students performed at each of the levels in the context of 
the expectations for achievement at each of the levels and cross-checking with the actual 
question-by-question results yields some interesting findings. In general, the sets of reading 
skills expected were those observed. However, in some instances, particularly for extended 
response questions, even Advanced-level students had difficulty providing in-depth answers. 
In some other instances, because the assessment was developed prior to the achievement 
level descriptions, particular reading skills were not measured. For more information about 
student performance, see the full report. 
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The following figure provides the percentage of students at or above each achievement 
level, as well as the percentage of students below the Basic level. In Utah, 64 percent of 
the fourth graders in public schools were at or above the Basic level, 26 percent were at 
or above the Proficient level, and 3 percent were at or above the Advanced level. 
Nationwide, 57 percent of the fourth graders were at or above the Basic level, 24 percent 
were at or above the Proficient level, and 4 percent were at or above the Advanced level. 
About the same percentage of students in Utah as across the nation were at or above the 
Proficient level. 



Levels of Fourth-Grade Public-School 
Students’ Reading Achievement 
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Percent 

3 ( 0 . 5 ) 

3 ( 0 . 5 ) 

4 ( 0 . 6 ) 



26 { 1 . 3 ) 
22 ( 1 . 6 ) 
24 ( 1 . 2 ) 



64 ( 1 . 5 ) 
53 ( 1 . 9 ) 
57 ( 1 . 2 ) 



36 ( 1 . 5 ) 
47 ( 1 . 9 ) 
43 ( 1 . 2 ) 



PERCENT 

The standard errors are presented in parentheses. With about 95 percent confidence, the 
average reading proficiency for each population of interest is within ± 2 standard errors of 
the estimated mean (95 percent confidence interval, denoted by W-l). If the confidence 
intervals for the populations do not overlap, there is a statistically significant difference 
between the populations. If they do overlap, the difference may or may not be statistically 
significant. Statistical tests comparing the two estimates must be conducted that use the 
standard error of the difference (see Appendix A for details). 
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PERFORMANCE ACCORDING TO PURPOSE FOR READING 

The 1992 Trial State Assessment Program considered students’ performance in situations 
that involved reading different kinds of materials for different purposes. The fourth-grade 
reading assessment measured two global purposes for reading - reading for literary 
experience and reading to gain information. Students in Utah performed higher than 
students across the nation in reading for literary experience and to gain information. 



Subpopulation Performance 



Assessment results repeatedly show differences in achievement for subpopulations of 
students. 7 The 1992 Trial State Assessment provides additional information about the 
achievement of important subpopulations by reporting on the performance of various 
subgroups of the student population defined by race/ethnicity, type of community, parents’ 
education level, and gender. 



Race/ Ethnicity 



White students in Utah demonstrated higher average reading proficiency than did 
Hispanic students. In Utah, about one quarter of the White students (28 percent) 
and some of the Hispanic students (11 percent) were at or above the Proficient 
level. Across the nation, about one quarter of the White students (30 percent) and 
some of the Hispanic students (12 percent) were at or above the Proficient level. 



Type of Community 



The average reading performance of Utah students attending schools in advantaged 
urban areas was higher than that of students attending schools in disadvantaged 
urban areas or areas classified as “other” and about the same as that of students 
attending schools in extreme rural areas. Less than half of the students attending 
schools in advantaged urban areas (35 percent), some of the students in 
disadvantaged urban areas (11 percent), about one quarter of the students in 
extreme rural areas (24 percent), and about one quarter of the students in areas 
classified as “other” (25 percent) in Utah were at or above the Proficient level. 
Across the nation, about half of the students in advantaged urban areas 
(47 percent), relatively few of the students in disadvantaged urban areas 
(5 percent), about one quarter of the students in extreme rural areas (24 percent), 
and about one quarter of the students in areas classified as “other” (24 percent) 
were at or above the Proficient level. 



7 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. Trends in 
Academic Progress. (Washington, DC: National Center for Education Statistics, 1992). 
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Parents’ Education 



Students in Utah who reported that at least one parent graduated from college 
demonstrated about the same average reading proficiency as did students who 
reported that at least one parent had some education after high school, but higher 
proficiency than did students who reported that at least one parent graduated from 
high school, neither parent graduated from high school, or they did not know their 
parents’ education level. Reading achievement in Utah was at or above the 
Proficient level for 34 percent of the students who reported that at least one parent 
graduated from college, 36 percent of the students who reported that at least one 
parent had some education after high school, 17 percent of the students who 
reported that at least one parent graduated from high school, 15 percent of the 
students who reported that neither parent graduated from high school, and 
18 percent of the students who reported that they did not know their parents’ 
education level. Across the nation, these figures were 33 percent of the students 
who reported that at least one parent graduated from college, 28 percent of the 
students who reported that at least one parent had some education after high 
school, 18 percent of the students who reported that at least one parent graduated 
from high school, 10 percent of the students who reported that neither parent 
graduated from high school, and 17 percent of the students who reported that they 
did not know their parents’ education level. 



Gender 



In Utah, fourth-grade boys attending public schools had a lower average reading 
proficiency than did fourth-grade girls. Compared to the national results, girls in 
Utah performed higher than girls across the country; boys in Utah performed 
higher than boys across the country. There was a significant difference between 
the percentages of males and females in Utah who attained the Proficient level 
(29 percent for females and 23 percent for males). The percentage of females in 
Utah who attained the Proficient level was about the same as the percentage of 
females in the nation who attained the Proficient level (29 percent for Utah and 
26 percent for the nation). Similarly, the percentage of males in Utah who 
attained the Proficient level was about the same as the percentage of males in the 
nation who attained the Proficient level (23 percent for Utah and 21 percent for 
the nation). 



A Context for Understanding Students’ Reading Proficiency 

Information on the reading performance of students in Utah can be better understood and 
used for improving instruction and setting policy when supplemented with contextual 
information about schools, teachers, and students. 

To gather contextual information, the fourth-grade students participating in the 1992 Trial 
State Assessment, their reading teachers, and the principals or other administrators in their 
schools were asked to complete questionnaires on policies, instruction, and programs. The 
student, teacher, and school data help to describe some of the current practices and 
emphases in reading education, illuminate some of the factors that appear to be related to 
fourth-grade public-school students’ reading proficiency, and provide an educational 
context for understanding information on student achievement. Highlights of the results 
for the public-school students in Utah are as follows: 
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CURRICULUM COVERAGE AND INSTRUCTIONAL EMPHASIS 

• In Utah, 62 percent of the fourth-grade students had reading teachers who 
spent at least 60 minutes providing reading instruction each day. By 
comparison, 38 percent of the students had reading teachers who spent 
45 minutes or less providing reading instruction each day. 

• In Utah, some of the fourth-grade students (13 percent) were being taught 
by teachers who placed heavy emphasis on phonics; about half 
(45 percent) were being taught by teachers who placed heavy emphasis on 
the integration of reading and writing; and less than half (34 percent) were 
being taught by teachers who placed heavy emphasis on the whole language 
approach. 

• In addition, in Utah, about half of the fourth-grade students (47 percent) 
were being taught by teachers who placed heavy emphasis on 
literature-based reading; less than half (40 percent) were being taught by 
teachers who placed heavy emphasis on reading across the content areas; 
and some (17 percent) were being taught by teachers who placed heavy 
emphasis on individualized reading programs. 



DELIVERY OF READING INSTRUCTION 

• More than half of the fourth-grade public-school students in Utah 
(57 percent) had reading teachers who used both basal and trade books, 
less than half (32 percent) had reading teachers who primarily used basal 
readers, and relatively few (8 percent) had reading teachers who primarily 
used trade books. 

• In Utah, 1 percent of the fourth-grade students had reading teachers who 
used children’s newspapers and/or magazines almost every day; 13 percent 
of the students had reading teachers who used reading kits almost every 
day; 5 percent had reading teachers who used computer software for 
reading instruction almost every day; 44 percent had reading teachers who 
used a variety of books almost every day; and, finally, 28 percent had 
teachers who used materials from other subject areas almost every day. 

• In Utah, some of the fourth-grade students (15 percent) had reading 
teachers who devoted almost all of their instructional time in reading to 
teaching decoding skills; about one quarter of the students (29 percent) had 
reading teachers who devoted almost all of their instructional time in 
reading to oral reading; less than half (31 percent) had reading teachers 
who devoted almost all of their instructional time in reading to teaching 
vocabulary; about half (53 percent) had reading teachers who devoted 
almost all of their instructional time in reading to 
comprehension/interpretation; and finally, about one quarter (23 percent) 
had teachers who devoted almost all of their instructional time in reading 
to reading strategies. 
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EDUCATIONAL BACKGROUND OF TEACHERS 

• In Utah, 19 percent of the students were being taught by reading teachers 
who reported having at least a master’s or education specialist’s degree. 
This compares with 46 percent for students across the nation. 

• More than half of the students (63 percent) had reading teachers who had 
the highest level of teaching certification that is recognized by Utah. This 
is about the same as* the figure for the nation, where more than half of the 
students (57 percent) were taught by reading teachers who were certified 
at the highest level available in their states. 

• In Utah, 26 percent of the fourth-grade public-school students were being 
taught reading by teachers who had an undergraduate major in English, 
reading, and/or language arts. By comparison, 22 percent of the students 
across the nation had reading teachers with the same major. 

HOME FACTORS 

• Students in Utah who had four types of reading materials in the home 
(newspapers, magazines, more than 25 books, and an encyclopedia) 
showed a higher reading proficiency than did students with zero to two 
types of materials. Across the nation, students who had all four types of 
materials at home showed a higher reading proficiency than did students 
who had zero to two types. 

• In Utah, 23 percent of the fourth-grade public-school students discussed 
what they read with friends or family almost every day; 23 percent never 
or hardly ever discussed what they read. Across the nation, 27 percent 
discussed what they read with friends or family almost every day and 
24 percent never or hardly ever discussed what they read. 

• About one quarter of the fourth-grade public-school students in Utah 
(28 percent) watched one hour or less of television each day; some 
(1 1 percent) watched six hours or more. 



Comparisons of Overall Reading Proficiency in Utah with Other 
States 

The map on the following page provides a method for making appropriate comparisons 
of the overall reading proficiency in Utah with that in other states (including the District 
of Columbia and one territory) that participated in the NAEP 1992 Trial State Assessment 
Program. The different shadings of the states on the map show whether the average overall 
proficiency in the other states was statistically different from or not statistically different 
from that in Utah (“Target State”). States with a dark-colored shading have a significantly 
higher average proficiency than does Utah. States with a light-colored shading have a 
significantly lower average proficiency than does Utah. States without shading have an 
average proficiency that does not differ significantly from that of Utah. The significance 
tests are based on a Bonferroni procedure for multiple comparisons that holds the 
probability of erroneously declaring the means of any two states to be different, when they 
are not, to no more than five percent. 



* Recall that “about the same” means that the difference between these two groups, although it may appear 
large, is not statistically significant. 
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OVERVIEW 



THE NATION’S 




Trial State Assessment 



In 1988, Congress passed new legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by-state assessments on a trial basis: 

The National Assessment shall develop a trial mathematics assessment survey 
instrument for the eighth grade and shall conduct a demonstration of the 
instrument in 1990 in States which wish to participate, with the purpose of 
determining whether such an assessment yields valid , reliable State representative 
dUta. (Section 406(i)(2)(C)(i) of the General Education Provisions Act, as 
amended by Pub . L. 100-297 (U.S.C. 1221e-l(i)(2)(c)(i))) 

The National Assessment shall conduct a trial mathematics assessment for the 
fourth and eighth grades in 1992 and, pursuant to subparagraph (6)(D), shall 
develop a trial reading assessment to be administered in 1992 for the fourth grade 
in States which wish to participate, with the purpose of determining whether such 
an assessment yields valid, reliable State representative data. (Section 
406(i)(2)(C)(i) of the General Education Provisions Act, as amended by Pub. 

L. 100-297 (U.S.C. 1 221e-l(i)(2)(c)(ii))) 



As a result of the legislation, the 1990 NAEP program included a Trial State Assessment 
Program in which public-school students in 37 states, the District of Columbia, and two 
territories were assessed in eighth-grade mathematics. 8 The 1992 NAEP program included 
an expanded Trial State Assessment Program in fourth-grade reading and fourth- and 
eighth-grade mathematics, with public-school students assessed in 41 states, the District of 
Columbia, and two territories. 9 In addition, national assessments in mathematics, reading, 
writing, and science were conducted concurrently with the Trial State Assessment Program 
in 1990 and in 1992. 



8 For a summary of the 1990 program, see Ina V.S. MuIIis, John A. Dossey, Eugene H. Owen, and Gary W. 
Phillips. The State of Mathematics Achievement : NAEPs J990 Assessment of the Nation and the Trial 
Assessment of the States. (Washington, DC: National Center for Education Statistics, 1991). 

9 For a summary of the 1992 assessment of mathematics, see NAEP 1992 Mathematics Report Card for the 
Nation and the States. (Washington, DC: National Center for Education Statistics, 1993). 
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The 1992 Trial State Assessment Program was conducted in February 1992 with the 
following 44 participants: 



Alabama 


Louisiana 


Ohio 


Arizona 


Maine 


Oklahoma 


Arkansas 


Maryland 


Pennsylvania 


California 


Massachusetts 


Rhode Island 


Colorado 


Michigan 


South Carolina 


Connecticut 


Minnesota 


Tennessee 


Delaware 


Mississippi 


- Texas , 


District of Columbia 


Missouri 


, • • Utah % ?•} 
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Nebraska > 


Virginia 


Georgia 


New Hampshire 


West Virginia ;/ 


Hawaii 


New Jersey 


Wisconsin v'^'S 


Idaho v*. 


New Mexico 




Indiana 


New York 


' Iowa 


North Carolina 


•' 5 ’ ‘V Guam r-vi 


Kentucky 


North Dakota 


Virgin Islands* 









* The Virgin Islands participated in the testing portion of the 1992 Trial State Assessment Program. However, 
in accordance with the legislation providing for participants to review and give permission for release of their 
results, the Virgin Islands chose not to release their results at grade 4 in the reports. 



States in regular type did not participate in the 1990 Trial State Assessment. Three states 
- Montana, Illinois, and Oregon -- participated in the 1990 Trial State Assessment but not 
in the 1992 program. 

For the 1992 Trial State Assessment in reading, approximately 2,500 students were assessed 
in each jurisdiction. The samples were carefully designed to represent the fourth-grade 
public-school populations in the states or territories. Similar to the 1990 program, local 
school district personnel administered all assessment sessions, and the contractor’s staff 
monitored 50 percent of the sessions as part of the quality assurance program designed to 
ensure that the sessions were conducted uniformly. The results of the monitoring in 1990 
and 1992 indicated a high degree of quality and uniformity across sessions. 

The 1992 Trial State and National Assessment programs in reading were based on a 
framework developed through a national consensus process that was set forth by law and 
called for “active participation of teachers, curriculum specialists, subject matter specialists, 
local school administrators, parents, and members of the general public” (Public Law 
100-297, Part C, 1988). 10 



10 Reading Framework for the 1992 National Assessment of Educational Progress. (Washington, DC: National 
Assessment Governing Board, U.S. Department of Education, 1992). 
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The process of developing the framework was carried out in late 1989 and early 1990 by 
the Council of Chief State School Officers (CCSSO) under contract from the National 
Assessment Governing Board (NAGB) which is responsible for formulating policy for 
NAEP, including developing assessment objectives and test specifications. The framework 
development process included input from a wide range of people in the fields of reading 
and assessment, from school teachers and administrators to state coordinators of reading 
and reading assessment. After thorough discussion and some amendment, the framework 
was adopted by NAGB in March 1990. An overview of the reading framework is provided 
in the Procedural Appendix. 

The fourth-grade Trial State and National Assessments in reading included eight sections 
or blocks, each 25 minutes in length. Each block consisted of a passage and a combination 
of constructed-response and multiple-choice questions. Passages selected for the 
assessment were drawn from authentic texts used by students in real, everyday reading. 
Whole stories, articles, or sections of textbooks were used, rather than excerpts or 
abridgements. The type of question -- constructed-response or multiple-choice -- was 
determined by the nature of the task. In addition, the constructed-response questions were 
of two types: regular constructed-response questions required students to respond to a 
question in a few words or a few sentences while extended constructed-response questions 
required students to respond to a question in a paragraph or more. 



This Report 



This is a computer-generated report that describes the reading performance of fourth-grade 
public-school students in Utah, in the West region, and across the nation. A separate 
report describes additional fourth-grade reading assessment results for the nation and the 
states, as well as the national results for grades 8 and 12. 11 This report consists of three 
sections: 



• This Overview provides background information about the Trial State 
Assessment and a profile of the fourth-grade public-school students in 
Utah. 

• Part One describes the reading performance of the fourth-grade 
public-school students in Utah, the West region, and the nation. 

• Part Two relates fourth-grade students’ reading performance to contextual 
information about the reading policies, instruction, and home support for 
reading in Utah, the West region, and the nation. 



11 See NAEP 1992 Reading Report Card for the Nation and the Slates. (Washington, DC: National Center for 
Education Statistics, 1993). 
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In this report, results are provided for groups of students defined by shared characteristics 
— race/ethnicity, type of community, parents’ education level, and gender. Definitions of 
the subpopulations referred to in this report are presented below. The results for Utah are 
based on the representative sample of public-school students who participated in the 1992 
Trial State Assessment Program. The results for the nation and the region of the country 
are based on the nationally and regionally representative samples of public- school students 
who were assessed in January through March as part of the national NAEP program. 
Using the regional and national results from the 1992 national NAEP program is necessary 
because of the voluntary nature of the Trial State Assessment Program. Since not every 
state participated in the program, the aggregated data across states did not necessarily 
provide representative national or regional results. Specific details on the samples and 
analysis procedures used can be found in the Technical Report of the 1992 NAEP Trial 
State Assessment Program in Reading } z 



RACE/ETHNICITY 

Results are presented for students of different racial/ethnic groups based on the students’ 
self-identification of their race/ethnicity according to the following mutually exclusive 
categories: White, Black, Hispanic, Asian (including Pacific Islander), and American 
Indian (including Alaskan Native). Based on criteria described in the Procedural Appendix, 
there must be at least 62 students in a particular subpopulation in order for the results for 
that subpopulation to be considered reliable. Thus, results for racial/ethnic groups with 
fewer than 62 students are not reported. However, the data for all students, regardless of 
whether their racial/ethnic group was reported separately, were included in computing 
overall results for Utah. 



TYPE OF COMMUNITY 

Results are provided for four mutually exclusive community types -- advantaged urban, 
disadvantaged urban, extreme rural, and other — as defined below: 

Advantaged Urban: Students in this group live in metropolitan statistical areas 
and attend schools where, according to their schools, a high proportion of the 
students’ parents are in professional or managerial positions. 

Disadvantaged Urban : Students in this group live in metropolitan statistical 
areas and attend schools where, according to their schools, a high proportion 
of the students’ parents are on welfare or are not regularly employed. 



12 Technical Report of the NAEP 1992 Trial State Assessment Program in Reading. (Washington, DC: 
National Center for Education Statistics, 1993). 
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Extreme Rural: Students in this group live outside metropolitan statistical 

areas, live in areas with a population below 10,000, and attend schools where, 
according to their schools, many of the students’ parents are farmers or farm 
workers. 

Other: Students in this category attend schools in areas other than those defined 
as advantaged urban, disadvantaged urban, or extreme rural. 



Indices were developed such that approximately 10 percent of the most extreme advantaged 
urban, disadvantaged urban, and rural schools sampled in the national assessment were 
classified into these three categories. The remaining 70 percent of the schools were 
classified into the “other” category. The reporting of results by each type of community 
was also subject to a minimum student sample size of 62. 



PARENTS’ EDUCATION LEVEL 

Students were asked to indicate the extent of schooling for each of their parents — did not 
finish high school, graduated from high school, some education after high school, or 
graduated from college. The response indicating the higher level of education was selected 
for reporting. Reporting of results by parents’ education level was also subject to a 
minimum student sample size of 62. Note that a substantial percentage of fourth-grade 
students did not know their parents’ education level. 



GENDER 

Results are reported separately for males and females. 



REGION 

The United States has been divided into four regions for purposes of this report: Northeast, 
Southeast, Central, and West. States included in each region are shown in Figure 1. All 
50 states and the District of Columbia are fisted, with the participants in the Trial State 
Assessment highlighted in boldface type. Territories were not assigned to a region. 
Further, the part of Virginia that is included in the Washington, DC, metropolitan 
statistical area is included in the Northeast region; the remainder of the state is included in 
the Southeast region. Because most of the Virginia students are in the Southeast region, 
regional comparisons for Virginia are to the Southeast. 

The regional results are based on a separate sample from that used to report the state 
results. Regional results are based on national assessment samples, not on aggregated Trial 
State Assessment samples. 
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FIGURE l | Regions of the Country 
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Guidelines for Analysis 

This report describes reading proficiency for fourth graders attending public schools and 
compares the results for various groups of students within that population — for example, 
those who have certain demographic characteristics or who responded to a specific 
background question in a particular way. The report examines the results for individual 
groups and individual background questions. It does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. 

Because the proportions of students in these subpopulations and their average proficiencies 
are based on samples — rather than the entire population of fourth graders in public schools 
in a state or territory — the numbers reported are necessarily estimates. As such, they are 
subject to a measure of uncertainty, reflected in the standard error of the estimate. When 
the proportions or average proficiencies of certain groups are compared, it is essential to 
take the standard error into account, rather than rely solely on observed similarities or 
differences. Therefore, the comparisons discussed in this report are based on statistical tests 
that consider both the magnitude of the difference between the means or proportions and 
the standard errors of those statistics. 
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The statistical tests determine whether the evidence - based on the data from the groups 
in the sample - is strong enough to conclude that the means or proportions are really 
different for those groups in the population. If the evidence is strong (i.e., the difference is 
statistically significant ), the report describes the group means or proportions as being 
different (e.g., one group performed higher than or lower than another group) - regardless 
of whether the sample means or sample proportions appear to be about the same or not. 
If the evidence is not sufficiently strong (i.e., the difference is not statistically significant), 
the means or proportions are described as being about the same - again, regardless of 
whether the sample means or sample proportions appear to be about the same or widely 
discrepant. The reader is cautioned to rely on the results of the statistical tests - rather than 
on the apparent magnitude of the difference between sample means or proportions -- to 
determine whether those sample differences are likely to represent actual differences 
between the groups in the population. The statistical tests and Bonferroni procedure, 
which is used when more than two groups are being compared, are discussed in greater 
detail in the Procedural Appendix. 

In addition, some of the percentages reported in the text of the report are given quantitative 
descriptions (e.g., some, about half, almost all, etc.). The descriptive phrases used and the 
rules used to select them are described in the Procedural Appendix. 

Finally, in several places in this report, results (mean proficiencies and proportions) are 
reported in the text for combined groups of students. For example, in the text, the 
percentage of students in the combined group where teachers reported spending 60 minutes 
or 90 minutes or more on reading instruction on a typical day is given and compared to the 
group where teachers reported spending 45 minutes or less. However, the table that 
accompanies that text reports percentages and proficiencies separately for the three groups 
(45 minutes or less, 60 minutes, and 90 minutes or more). The combined group 
percentages reported in the text and used in all statistical tests are based on unrounded 
estimates (i.e., estimates calculated to several decimal places) of the percentages in each 
group. The percentages shown in the tables are rounded to integers. Thus, percentages 
may not always add up to 100 percent due to rounding. Also, the percentage for a 
combined group (reported in the text) may differ slightly from the sum of the separate 
percentages (presented in the tables) for each of the groups that were combined. Therefore, 
if statistical tests were to be conducted based on the rounded numbers in the tables, the 
results might not be consonant with the results of the statistical tests that are reported in 
the text (based on unrounded numbers.) 
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Profile of Utah 

FOURTH-GRADE SCHOOL AND STUDENT CHARACTERISTICS 



Table 1 provides a profile of the demographic characteristics of the fourth-grade 
public-school students in Utah, the West region, and the nation. This profile is based on 
data collected from the students and schools participating in the 1992 Trial State and 
National Assessments. As described earlier, the state data and the regional and national 
data are drawn from separate samples. 
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DEMOGRAPHIC SUBGROUPS 



Race/Ethnicity 

White 

Black 

Hispanic 

Asian 

American Indian 

Type of Community 

Advantaged urban 
Disadvantaged urban 
Extreme rural 
Other 

Parents’ Education 

Graduated college 
Some education after high school 
Graduated high school 
Did not finish high school 
I don't know 

Gender 

Male 

Female 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that 
for each population of interest, the value for the entire population is within ± 2 standard errors of the estimate 
for the sample. In comparing two estimates, one must use the standard error of the difference (see 
Appendix A for details). The percentages for Race/Ethnicity may not add to 100 percent because some 
students categorized themselves as “Other.” 



SCHOOLS AND STUDENTS ASSESSED 

Table 2 summarizes participation data for Utah schools and students sampled for the 1992 
Trial State Assessment. 13 In Utah, 108 public schools participated in the fourth-grade 
reading assessment. These numbers include participating substitute schools that were 
selected to replace some of the nonparticipating schools from the original sample. The 
weighted school participation rate was 99 percent, which means that the fourth-grade 
students in this sample of schools were directly representative of 99 percent of all the 
fourth-grade public-school students in Utah. 
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3 For a detailed discussion of the NCES guidelines for sample participation, see School and Student 
Participation Rates for the Reading Assessment. (Washington, DC: National Center for Education Statistics, 
1993) or see Appendix B of the Technical Report of the NAEP 1992 Trial State Assessment Program in 
Reading. (Washington, DC: National Center for Education Statistics, 1993). 
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In each school, a random sample of students was selected to participate in the assessment. 
As estimated by the sample, 1 percent of the fourth-grade public-school population was 
classified as Limited English Proficient (LEP), while 9 percent in fourth grade had an 
Individualized Education Plan (IEP). An IEP is a plan, written for a student who has been 
determined to be eligible for special education, that typically sets forth goals and objectives 
for the student and describes a program of activities and/or related services necessary to 
achieve the goals and objectives. Handicapped or disabled students may be categorized 
as IEP. 

Schools were permitted to exclude certain students from the assessment, provided that 
certain criteria were met. To be excluded, a student had to be categorized as Limited 
English Proficient or had to have an Individualized Education Plan and (in either case) be 
judged incapable of participating in the assessment. The intent was to assess all selected 
students; therefore, all selected students who were capable of participating in the assessment 
should have been assessed. However, schools were allowed to exclude those students who, 
in the judgment of school staff, could not meaningfully participate. The NAEP guidelines 
for exclusion are intended to assure uniformity of exclusion criteria from school to school. 
Note that some LEP and IEP students were deemed eligible to participate and not excluded 
from the assessment. The students in Utah who were excluded from the assessment 
because they were categorized as LEP or had an IEP represented 5 percent of the 
population in grade four. 

In total, 2,829 fourth-grade Utah public-school students were assessed. The weighted 
student participation rate was 96 percent. This means that the sample of students who 
took part in the assessment was directly representative of 96 percent of the eligible 
fourth-grade public-school student population in participating schools in Utah (that is, all 
students from the population represented by the participating schools, minus those students 
excluded from the assessment). 

The overall weighted response rate (school rate times student rate) was 95 percent. This 
means that the sample of students who participated in the assessment was directly 
representative of 95 percent of the eligible fourth-grade public-school population in Utah. 

In the analysis of student data and reporting of results, nonresponse weighting adjustments 
have been made at both the school and student level, with the aim of making the sample 
of participating students as representative as possible of the entire eligible fourth-grade 
public-school population. For details of the nonresponse weighting adjustment 
procedures, see the Technical Report of the NAEP 1992 Trial State Assessment Program in 
Reading. 
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TABLE 2 



Profile of the Fourth-Grade 
Population Assessed in Utah 
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How Proficient in Reading are Fourth-Grade 
Students in Utah Public Schools? 



Reading involves the interaction between a reader, a text, and a situation. 14 Thus, 
students’ reading comprehension is influenced by the type of material read and the specific 
purposes for reading. The 1992 Trial State Assessment Program considered students’ 
performance in situations that involved reading different kinds of materials for different 
purposes. The fourth-grade reading assessment measured two global purposes for reading 
-- reading for literary experience and reading to gain information . 15 Reading for literary 
experience usually involves the reading of novels, short stories, plays, and essays. In these 
reading situations, the reader can determine how the author explores experiences through 
the text and can consider the interplay among events, emotions, and possibilities. Reading 
to gain information usually involves the reading of articles in magazines and newspapers, 
chapters in a textbook, entries in encyclopedias and catalogs, and entire books on particular 
topics. These reading situations call for different orientations to text from those in reading 
for literary experience because readers are specifically focused on acquiring information. 
Students’ performance on each of the two purposes for reading was summarized on 
separate NAEP reading scales (one for each purpose), which range from 0 to 500. In 
addition, an overall reading scale, reflecting combined performance in the two purposes for 
reading, was also developed. The overall reading scale also ranges from 0 to 500. 
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14 Judith A. Langer, Arthur N. Applebee, Ina V.S. Mullis, and Mary A. Foertsch. Learning to Read in Our 
Nation's Schools. (Princeton, NJ: National Assessment of Educational Progress, Educational Testing 
Service, 1990). 

15 The eighth- and twelfth-grade national NAEP reading assessments also measured a third purpose for reading 
- reading to perform a task. 
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This part of the report contains two chapters that describe the reading proficiency of 
fourth-grade public-school students in Utah. Chapter 1 compares the overall reading 
performance of the students in Utah to students in the West region and the nation. It also 
presents the students’ average proficiency for the two purposes for reading. Chapter 2 
summarizes the students’ overall reading performance, as well as performance for each of 
the two reading purposes, for subpopulations defined by race/ethnicity, type of community, 
parents’ education level, and gender. 
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CHAPTER 1 



Students’ Reading Performance 



Reading achievement is central to one of the goals adopted by the president and the 



grades four , eight , and twelve having demonstrated competency in challenging subject matter 
including English , mathematics, science, history, and geography ; and every school in America 
will ensure that all students learn to use their minds well, so they may be prepared for 
responsible citizenship, further learning, and productive employment in our modern 



students of all ages have difficulty reading thoughtfully. 17 

Reading for meaning involves a dynamic, complex interaction between and among the 
reader, the text, and the context. Readers, for example, bring to the process their prior 
knowledge about the topic, their reasons for reading it, their individual reading skills and 
strategies, and their understanding of differences in text structures. 

The texts used in the reading assessment are representative of common reading demands. 
Students in grade 4 are asked to respond to literary and informational texts which differ in 
structure, organization, and features. Literary texts include short stories, poems, and plays 
that engage the reader in a variety of ways, not the least of which is reading for fun. 
Informational texts include selections from textbooks, magazines, encyclopedias, and other 
written sources whose purpose is to increase the reader’s knowledge. 



16 AMERICA 2000: An Education Strategy (Washington, DC: U.S. Department of Education, 1991). 

17 Ina V.S. Mullis, John A. Dossey, Mary A. Foertsch, Lee R. Jones, and Claudia A. Gentile. Trends in 
Academic Progress. (Washington, DC: National Center for Education Statistics, 1992); Judith A. Langer, 
Arthur N. Applebee, Ina V.S. Mullis, and Mary A. Foertsch. Learning to Read in Our Nation’s Schools. 
(Princeton, NJ: National Assessment of Educational Progress, Educational Testing Service, 1990); Richard 
C. Anderson, Elfrieda H. Hiebert, Judith A. Scott, Ian A. Wilkinson. Becoming a Nation of Readers. 
(Washington, DC: National Institute of Education, 1985). 



governors following the historic Charlottesville conference — American students will leave 



economy }* Concern about attaining the goal and, more importantly, about the reading 
abilities of our nation’s students has increased recently because it appears that many 
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The context of the reading situation includes the purposes for reading that the reader might 
use in building a meaning of the text. For example, in reading for literary experience, 
students may want to see how the author explores or uncovers experiences, or they may 
be looking for vicarious experience through the story’s characters. On the other hand, the 
student’s purpose in reading informational texts may be to learn about a topic (such as the 
Civil War or the oceans) or to search for specific information. 



The assessment asks students to build, extend, and examine text meaning from four stances 
or orientations: 



Initial 

Understanding 



Students are asked to provide the overall or general meaning of the selection. This 
includes summaries, main points, or themes. 



Developing an 
Interpretation 



Students are asked to extend the ideas in the text by making inferences and 
connections. This includes making connections between cause and effect, 
analyzing the motives of characters, and drawing conclusions. 



Personal 

Response 



Students are asked to make explicit connections between the ideas in the text and 
their own background knowledge and experiences. This includes comparing story 
characters with themselves or people they know, for example, or indicating 
whether they found a passage useful or interesting. 



Critical Stance 



Students are asked to consider how the author crafted a text. This includes 
identifying stylistic devices such as mood and tone. 



These stances are not considered hierarchical or completely independent of each other, but 
are iterative. They provide a frame for generating questions and considering student 
performance at all levels. All students at all levels should be able to respond to reading 
selections from all of these orientations. What varies with students’ developmental and 
achievement levels is the amount of prompting or support needed for response, the 
complexity of the texts to which they can respond, and the sophistication of their answers. 

As shown in Figure 2, the overall average proficiency of fourth-grade public- school 
students from Utah on the NAEP reading scale was 222. This proficiency was higher than 
that of students across the nation (216). 18 There also was a tremendous range in student 
performance as shown by the percentiles of the distribution of reading proficiency in Utah 
presented in Table 3. The lowest performing 10 percent of the fourth graders from Utah 
had proficiency levels below 180 while the top 10 percent had proficiency levels above 260. 



18 Differences reported as significant are statistically different at the 95 percent confidence level. This means 
that with 95 percent confidence there is a real difference in the average reading proficiency between the two 
populations of interest. “About the same” means that no statistically significant difference was found at the 
95 percent confidence level. 
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FIGURE 2 



Fourth-Grade Public-School Students’ 
Average Reading Proficiency 
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The NAEP reading scale ranges from 0 to 500. The standard errors are presented in parentheses. With about 
95 percent confidence, the average reading proficiency for each population of interest is within ± 2 standard 
errors of the estimated mean (95 percent confidence interval, denoted by h4H). If the confidence intervals for 
the populations do not overlap, there is a statistically significant difference between the populations. If they do 
overlap, the difference may or may not be statistically significant. Statistical tests comparing the two estimates 
must be conducted that use the standard error of the difference (see Appendix A for details). 
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TABLE 3 



Percentiles of Reading Proficiency 
for Fourth-Grade Public-School 
Students 



5th 


10th 


25th 


50th 


75th 


90th 


95th 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 




The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It 
can be said with about 95 percent confidence that, for each population of interest, the value for the entire 
population is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must 
use the standard error of the difference (see Appendix A for details). 
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LEVELS OF READING ACHIEVEMENT 

Average proficiency on the NAEP scale provides an overall depiction of students’ reading 
achievement; however, by itself, it does not describe what students know and are able to 
do, nor does it evaluate student performance against a standard. This report next presents 
a set of results based on applying the National Assessment Governing Board’s standards 
to student performance on the reading scale. 

When Congress established the National Assessment Governing Board (NAGB) in 1988 
to set policy for NAEP, it charged the board with “identifying appropriate achievement 
goals for each age and grade in each subject area to be tested under the National 
Assessment.” (Pub.L. 297-100, Section 3403 (a)(5)(B)(ii)). To carry out this responsibility, 
NAGB contracted with American College Testing (ACT) to undertake advisory and 
analytic functions that could assist the Board in forming its conclusions as to appropriate 
achievement levels to be used for evaluating the 1992 reading assessment results. 
Achievement levels are mappings of collective judgments about how students should 
perform onto the achievement scale. 19 Boundary points were developed for three 
achievement levels for each grade — Basic, Proficient, and Advanced. Performance at the 
Basic level denotes partial mastery of the knowledge and skills that are fundamental for 
proficient work at the fourth-grade level. The central level, called Proficient, represents 
solid academic performance at the fourth-grade level. Students reaching this level 
demonstrate competency over challenging subject matter and are well prepared for the next 
level of schooling. Achievement at the Advanced level signifies superior performance in the 
fourth grade. 

This report follows NAGB’s policy that achievement levels should be the primary and 
initial method of presenting the results of the 1992 Trial State Assessment. In this report, 
these achievement levels are applied to the 1992 data, showing the proportions of students 
that achieved the three achievement levels. 

Definitions of the three levels of reading achievement are given in Figure 3. Examples of 
items at the achievement levels are provided. The reading passage which accompanies these 
items can be found in Appendix B. It should be noted that constructed-response items 
occur at all levels of reading achievement. 
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19 Appendix C briefly describes the process of gathering expert judgments about Basic, Proficient, and 
Advanced performance - as defined by NAGB policy - on each reading item, combining the various 
judgments on the various items and mapping them onto the scale, and setting the scale score cutpoints for 
reporting purposes based on these levels. 
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FIGURE 3 



Levels of Reading Achievement at Grade 4 
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The following achievement-level descriptions focus on the interaction of the reader, the 
text, and the context. They provide some specific examples of reading behaviors that 
should be familiar to most readers of this document. The specific examples are not 
inclusive; their purpose is to help clarify and differentiate what readers performing at each 
achievement level should be able to do. While a number of other reading achievement 
indicators exist at every level, space and efficiency preclude an exhaustive listing. It should 
also be noted that the achievement levels are cumulative from Basic to Proficient to 
Advanced. One level builds on the previous levels such that knowledge at the Proficient 
level presumes mastery of the Basic level, and knowledge at the Advanced level presumes 
mastery at both the Basic and Proficient levels. 



BASIC 

LEVEL 

( 212 ) 



Fourth-grade students performing at the Basic level should demonstrate an 
understanding of the overall meaning of what they read. When reading texts 
appropriate for fourth graders, they should be able to make relatively obvious 
connections between the text and their own experiences. 



Specifically, when reading literary text, they should be able to tell what the story is generally about — providing 
details to support their understanding — and be able to connect aspects of the stories to their own experiences. 

When reading informational text, Basic-level fourth graders should be able to tell what the selection is generally 
about or identify the purpose for reading it; provide details to support their understanding; and connect ideas 
from the text to their background knowledge and experiences. 



Fourth-grade students performing at the Proficient level should be able to 
demonstrate an overall understanding of the text, providing inferential as well 
as literal information. When reading text appropriate to fourth grade, they 
should be able to extend the ideas in the text by making inferences, drawing 
conclusions, and making connections to their own experiences. The connection 
between the text and what the student infers should be clear. 



Specifically, when reading literary text, Proficient-level fourth graders should be able to summarize the story, 
draw conclusions about the characters or plot, and recognize relationships such as cause and effect. 

When reading informational text, Proficient-level students should be able to summarize the information and 
identify the author’s intent or purpose. They should be able to draw reasonable conclusions from the text, 
recognize relationships such as cause and effect or similarities and differences, and identify the meaning of the 
selection’s key concepts. 



PROFICIENT 

LEVEL 

(243) 



ADVANCED 


Fourth-grade students performing at the Advanced level should be able to 
generalize about topics in the reading selection and demonstrate an awareness 


LEVEL 


of how authors compose and use literary devices. When reading text 


(275) 


appropriate to fourth grade, they should be able to judge texts critically and, in 


general, give thorough answers that indicate careful thought. 
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Specifically, when reading literary text, Advanced- level students should be able to make generalizations about 
the point of the story and extend its meaning by integrating personal and other reading experiences with the 
ideas suggested by the text. They should be able to identify literary devices such as figurative language. 



When reading informational text, Advanced-level fourth graders should be able to explain the author’s intent by 
using supporting material from the text. They should be able to make critical judgments of the text (including 
its form and content) and explain their judgments clearly. 
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FIGURE 3 
(continued) 



Levels of Reading Achievement at Grade 4 




Trial State Assessment 



The three items that follow were selected to exemplify each of the three achievement levels 
at grade 4. These items are all based on the story “Sybil Sounds the Alarm,” which is 
shown in its entirety in Appendix B. This is an historical narrative demonstrating the 
purpose “reading for literary experience.” For the multiple -choice items, the correct answer 
is marked with an asterisk. For the short constructed-response item, the scoring guide is 
provided. Also shown is the percent correct (conditional p-value) for the students 
performing within the interval of the indicated level. 







Percent Correct for Basic Interval 



Nation 



H > 76(2.4) Jg 



Sybil’s father thought that she 

A. was obedient but forgetful 
+ B. was courageous and a good rider 

C. could lead the troops against the British 

D. could easily become angry 





PROFICIENT LEVEL 
Example Item 


Percent Correct for Proficient Interval 


Nation 





The information about the statue and stamp helps to show that 
+ A. people today recognize and respect Sybil’s bravery 

B. people were surprised that George Washington honored her 

C. the author included minor details 

D. heroes are honored more now than they were then 
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